Methods
From Research Design to Methods: Explaining the How and Why
Methods Section of the Article
What to include
The methods section should include answers to the following questions:
- How did you do your research?
- What was the research design?
- Was it quantitative or quantitative?
Introduction
The Research Design
The Research Design = the overall plan to answer a research question.
It forms the backbone of the methods (tools) sections of a paper.
- What data did you use and how was it collected?
- What are your key variables and how were they measured?
- What analytical methods did you apply?
A good research design tries to ensure high internal and external validity
The Research Design
Internal Validity
Internal Validity - the extent to which we can be confident that the independent (causal) variable produced the observed effect.
- Is the independent variable responsible for variation in the dependent variable?
- What other possible causes might there be for the relationship between the variables?
- Could something elsehave been responsible for the variation in the dependent variable?
- Could there be confounding factors?
The Research Design
External Validity
External Validity - the extent to which the results from a study can be generalized beyond the particular study
- Can you generalize your findings?
- Are your conclusions likely to apply more widely?
Methods
Research methods = data collection + data analysis.
Approaches:
- Quantitative approaches - usually good at making generalizations: large-N
- Qualitative approaches - good at investigate these hard-to-define concepts and hard-to-reach populations: small-N
Data sources include:
- already existing quantitative data
- experiments
- surveys
- interviews or focus groups
- comparative research
- archival data and documentary records (e.g., speeches, policy documents)
Units of analysis (observations) can include people, countries, organizations, texts, etc.
Types of Research Designs
General Overview
Research designs can be classified in various ways. Common types include:
- Comparative designs – comparisons across countries, regions, or groups using observational data
- Exploratory (e.g., fieldwork, open-ended interviews)
- Explanatory/Causal (e.g., experiments, quasi-experiments, regression designs)
- Cross-sectional vs. longitudinal
- Quantitative vs. qualitative
Types of Research Designs
Focus of This Course
The ones that we will focus on are:
1.Comparative Analysis - qualitative
2.Experimental Designs - quantitative
3.Quasi-natural Experiments - quantitative
Comparative Analysis
Case Study: Intro
A case study focuses on the analysis on one country, event, or organization.
It must be situated comparatively to matter beyond itself.
Case studies:
- apply existing theory to new contexts;
- examine exceptions to the rule
- generate new theory
- have to explain how they are applicable to other contexts
- explore a causal process or mechanisms (process tracing)
Comparative Analysis
Case Study: Data
Case studies may use
- interviews
- surveys
- ethnography
- focus groups
- documents: policy documents and speeches
Comparative Analysis
Case Study: Examples
Voting behavior in Britain
Public Attitudes towards the environment in Germany
Public attitudes towards immigrants and ethnic minorities in the Netherlands.
Comparative Analysis
Comparative Designs: Intro
Comparative designs can be used:
- to apply existing theory to new cases;
- to develop new theory or hypotheses
- to test theory
Small-N studies are sometimes used for:
Process Tracing - uncover causal paths and mechanisms - assess specific mechanisms identified in theories.
Comparative Analysis
Comparative Designs: Small-N Comparison
Small-N Comparison involves the the comparison of two or more cases
- they offer detailed in-depth analysis of the case study,
- they provide greater scope for contextualization.
They can be based on the most similar research design:
- selecting countries that share many (theoretically) important characteristics, but differ in one crucial respect (related to the hypothesis of interest).
They can be based on the most dissimilar research design:
- cases that are different in most respects and only similar on the key explanatory variable of interest.
Comparative Analysis
Comparative Designs: Small-N Comparison
Examples:
- Michael Lewis-Beck’s (1986) economic voting in Britain, France, Germany, and Italy
- Seymour Martin Lipset’s (1959) study of the social requisites of democracy in Europe and South America;
- Thelda Skocpol’s (1979) study of revolution in Russia, France, and China;
Types of Research Designs
Experimental Designs
Most rigorous way to test if X causes Y. Why?
- You control who gets treatment vs. control
- Randomization eliminates confounders
Basic Steps
- Split into treatment and control groups
- Randomly assign participants
- Measure outcome before the intervention (pre-test)
- Apply the treatment
- Measure outcome again (post-test)
Types of Research Designs
Natural Experiments
You don’t control the treatment — but nature helps you out.
Key designs:
- differences-in-differences
- regression-discontinuity design
These methods try to mimic experiments, even without full control.
Types of Research Designs
Natural Experiments: Assumptions
Quasi-experiments need:
- Variation in the causal variable that’s independent of confounders
- That variation must be as good as random
Hard to guarantee in practice — strong designs are needed.
Note: These designs often have high internal validity, but limited generalizability (lower external validity)
Methods
Using the Model Article
Let us examine the model article that we identified
Depending on the method you use you will have to answer different questions
Methods
To Do 1
Get your draft under the methods section the answer to the following questions:
- what data are you using
- where is the data from
- how novel is the data
- how does the data help you answer the research question
Methods
To Do 2
Get your draft type under the methods section the answer to the following questions:
- how are you analyzing the data
- how are you interpreting your data
- are there any limitations to your data
Methods: Quantitative Methods
To Do 3
If you use a quantitative research design, you should also answer these questions
- What variables did you analyze?
- Why did you choose those variables?
Popescu (JCU): Methods